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ABSTRACT 

The construct validity of a multidimensional 
adolescent self-concept was estimated separately for males ana 
females, and the results were compared across gender. The application 
of four multitrait-multimethod (MTMM) approaches to estimating 
construct validity was demonstrated, and the scope and consistency of 
the findings derived from each were compared. Methodological 
approaches included: (1) the Campbell-Fiske model of D. T. Campfcell 
and D. W. Fiske (1959); (2) the analysis of variance model of M. J. 
Kavanagh et al. (1971); (3) the confirmatory factor analysis model of 
K. G. Joreslcog (1969); and (4) the composite direct product model of 
M. W. Browne (1984). These procedures were applied to data comprising 
four self-concept traits (general, academic, English, and 
mathematics) as measured by three measurement scales (Likert, 
semantic differential, and Cattman) . The sample consisted of 412 
males and 420 females in grades 11 and 12 from two suburban high 
schools in Ottawa (Ontario) . Results derived from the composite 
direct product model are less than optimal, other results demonstrate 
that adolescent males and females can differ with respect to various 
aspects of construct validity related to multidimensional facets of 
self-concept. Seven tables present study data. (SLD) 
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Abstra'' t 



The purposes of the study were (a) to estimate the construct 
validity of a multidinensional adolescent self-concept 
separately for males and females and to compare the restilts 
across gender, and (b) to demonstrate the application cf four 
MTMM approaches to estimatinp, construct validity 3nd to conpare 
the scope and consistency of the findings derived from each. 
Methodological approaches included the Campbel 1-Fi ske model 
(1959), the ANOVA model (Kavanagh, MacKinney, S Wolens, 1971), 
the confirnatory factor analysis model (Joreskog, 1969), and 
the composite direct product nodel (Browne, 1984). These 
procedures were applied to dfta comprising four self --concept 
traits (general, academic, English, mathematics) as measured by 
three measurement scales (Likert, semantic differential, 
Guttman) for 832 grades 11 and 12 high school students (males = 
412; females = 420). 
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Four Miiltitrait-mul t imethod Approaches to Exanininf^, the 
Construct Validity of Adolescent Self-concept 
Within and Across Gender 

The purposes of the present study were twofold. First,' to 
examine the construct validity of adolescent self --concept (SC) 
within gender and to compare the results across gender. Second, 
to demonstrate the application of four multitrait- miiltimethod 
(MMTM) approaches to determining construct validity and to 
compare the scope and consistency of the findings derived from 
each • 

Of late, an increasing number of substantive studies have 
investigated the possibility of gender differences in 
adolescent SC. In particular, interest has focused on general 
SC, academic SC, and the more specific subject-matter SCs 
related to mathematics and English. An important assumption in 
testing for mean differences in SC, however, is the equivalency 
across groups of (a) the structure of the underlying construct, 
and (b) the validity and reliability of the measures. 

Substantive research fi^idings bearing on gender differences 
in adolescent SC are, in general, inconsistent (for reviews see 
Byrne, 1984; West, Fish Stevens, 1980; Wylie, 1974, 1979). 
Although findings related to the more specific facets of 
English SC and mathematics SC are somewhat nore consistent in 
demonstrating higher English SCs for girls, albeit higher 
mathematics SCs for boys (Byrne ^ Shavelson. 19vS6; Itarsh, 
Parker Barnes, 1985; Meece, Parson, Kaczala, Goff R 
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Fetterman, 1982), there is still some tincertainty concernin<^ 
the equivalence of relations among these facets of the 
construct. Byrne and Shavelson (1987), in explaining the 
discrepant SC findings, cited three major limitations of 
earlier studies: (a) tho lack of a clear theoretical rationale, 

(b) the use of psychometric.'^lly limited instrumentation, and 

(c) the use of simplistic or inappropriate methodological 
procedures . 

The inconsistencies in SC research, however, can also be 
explained by gender differences in the construct validity of 
both the multidimensional ?;C facets, and the instruments 
designed to measure then. Indeed, implicit in multigroup 
comparisons is the assu.nption of group-invariant construct 
/alidit*'; nonet!ieless , such equivalence is '"arely tested 
directly. Although Byrne and Shavelson (1987) investigated 
invariance of the measurement and structure of adolescent SC, 
they did not examine the equivalence of convergent and 
discriminant validities nor mothod effects associated with the 
scale format of particular measuring instruments (seo Fiske, 
1987; Spector, 1937). 

Campbell and Fiske (1959) posited that claims of construct 
validity must be accompanied by evidence of both convergent and 
discriminant validity (see also, tlessick, 1981). As such, a 
measure should correlate highly '/ith other measures to which it 
is theoretically linked (convergent validity), and correlate 
negligibly with those that are theoretically unrelated 
(discriminant validity). Method effects represent bias that can 
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derive from use of the same ineasureTient method in the assess- 
ment of different constructs. Such bias impinges on the 
magnitude of trait relations, thus rendering results spurious. 
Method bias has been shown to be differentially influenced by 
gender (Hamilton, 1981), One procedure for determining evidonce 
of convergent and discriminant validity, and for detecting', 
method bias -s to examine data within a f!TMM matrix framework. 
The seminal work of Canpbell and Fiske (1959) provided the 
initial means to examining evi(!ence of construct validity. 
These researchers proposed that measures of multiple traits be 
assessed by nultiple methods nnd that all trait-method 
correlations be arranged in a MTMM matrix. The assessment of 
construct validity then focuses on comparisons among three 
blocks of correlations: (a) scores on the same traits measured 
by different methods ( monot rai t-het erone t hod values i,e,, 
convergent validity), (h) scores on different traits measured 
by the same method (heterotrait-mononethod values i,e,, 
discriminant validity) and, (c) scores op different traits 
neasured by different methods ( he tor ot ra i t-het ero-iot 'lOi] valMcs 
i,e,, discriminant validity). Specific criteria ^;uifie tho 
inspection of these values. 

Over the past decade, researchers have noted several 
shortcomings in the Campbel 1-Fi ske procedure and have proposed 
alternative approaches to the analysis of MT'^M matrices, '^ne 
such methodology summarizes tho MTMM data within an nnalysis of 
variance (ANOVA) three-way (subjGCt.s x traits x methods) 
^acworial design (Kavanagh, Macfvinney, ft V/olens, 1971), As 
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such, four major effects are of interest, and relate to 
variance due to: (a) subjects (convergent validity), (b) 
subjects by* traits (discriminant validity), (c) subjects by 
methods (method bias), and (d) error; all effects are tested 
statistically . 

A third approach to determining construct validity is the 
analysis of covariance structures using the confirmatory factor 
analytic (CFA) methodology proposed by Joreskog (1969). This 
procedure is purported to have several advantages over the 
Campbell-Fiske approach. First, the MTMM matrix is explained in 
terms of the underlying latent constructs, rather than the 
observed variables. Second, the evaluation of convergent and 
discriminant validities can be made at both the matrix and 
individual parameter levels. Third, h>potheses related to 
convergent and discriminant validity ca^ be testefl statis- 
tically, based on a series of hierarchically nested models. 
Finally, separate estimates of variance due to traits, me!:hods, 
and error /uniquenesses are provided. 

A fourth approach to determining construct validity derives 
from the decomposition of MTMM matrices using the Composite 
Direct Product (CDP) model proposed by Browne (1984). This 
methodology, as with the CFA model, is based on the anlaysis of 
covariance structures. Unlike the CFA model, however, the CDP 
model is based on a conponent analytic, rather then on a factor 
analytic methodological approach. This procedure conprises a 
more parsimonious parameterization of general trait and nethcd 
variance than the CFA model and, as such, is less subject to 
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problems of indeterminacy (Wothke, 1987). 

The present study extends the work of Byrne and Shavelson 
(1987) by reexamining the sane data within a MT?1M framework 
using four conceptually different approaches to the data 
analyses; each procedure providing more information and being a 
more powerful test of hypotheses than its predecessor. 
Specifically, the study examines the extent to which the 
convergent and discriminant validity of four SC traits (general 
SC, academic SC, English SC, mathematics SC), and the method 
effects associated with three measurement scales (Likert, 
semantic differential, Guttman) are equivalent across gend(^r, 
MTMM analyses will include evaluations based on (a) Campbell- 
Fiske criteria, (b) ANOVA nodel (Kavanagh, MacKinney ft Wolins, 
1971), (c) CPA model (Joreskog, 1969), and (d) CDP model 
(Browne, 1984) . 

Method 

Sample and Procedure 

Subjects were 832 (412 males, 420 females) grades 11 and 12 
students from two suburban high schools in Ottawa, Ontario. \ 
battery of SC instruments were adniinistered to intact classroom 
groups during one 50-minute period. Testing was completed 
approximately two weeks following the April report cards. 
Students therefore had the opportunity of beinf^, fully cognizant, 
of the ir academic performance prior to completin<> the tests for 
the study; this factor was considered important in moasurin^ 
academic and sub ject-'natter SCs. 
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Instrumentation 

The SC battery consisted of 12 neasuros; three different 
measurement' scales (Likert, semantic .1 i f f eren t ia 1 , Guttman) for 
each of general SC, acadenic SC, English SC, and mathematics 
SC. All were self-report instruments desif^ned specifically for 
use with a high school popul?tion. The Likert scales comprised 
subscales of the Self Description Questionnaire III (SDQ III; 
Marsh & O'Neill, 1984^ designed to measure general SC, academic 
SC, English SC and mathenatics SC; the semantic differential 
measures included subscales of the Affective Perception 
Inventory (API; Soares & Soares, 1979) designed to measured 
these same SC facets. Finally, the Guttman scales were the 
Self-esteem Scale (SES; l^osenberg, 1965) designed to measure 
general SC, and the Self-concept of Ability Scales (SCAS; 
Brookover, 1962) adapted for separate measurements of academic 
SC, and the subject-matter SCs. 
Analysis of the Data 

The data were analyzed separately for each MTMM model. 
First, zero-order correlations among all measurements were 
commuted and arranged in a separate MTMM matrix for males and 
for females; Campbell-Fiske criteria were used to assess 
evidence of convergent and discriminant validity both within 
and across gender. Second, trait-mithod correlations were 
decomposed into variance components and sunmari?ed within an 
ANOVA framework; variance explaining trait and nethod factors 
was compared across gender. Finally, using LISREL covariance 
factor (CFA model) and covariance component (CDP model) 
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analytic procedures, a 7-factor (4 traits, 3 methods) Tiodel of 
the data wa6 proposed and tested, separately, for males and 
females; estimates of ariance due to each trait and method 
were compared across fjender. 

Results 

Campbell-Fiske Criteria 

Campbell and Fiske (1959) proposed four criteria for 
evaluating convergent and discrininant validity. These criteria 
ar e : 

1. The convergent validities should be sif^nif icantly 
different from zero and sufficiently large to warrant further 
investigation of validity. 

2. The convergent validities should be higher than 
correlations between different traits assessed by different 
methods ( heterot rai t-heteromethod blocks) . 

3. The convergent validities should be higher than 
correlations between different traits assessed by the same 
method ( heterot rait-monomethod blocks) . 

4. The pattern of correlations between different traits 
should be the same in both the heteromethod and monoriethoci 
blocks . 

For each sex, comparisons of various blocks of correlations 
involved determining the proportion of times that these 
criteria were satisfied; these matrices of zero-order 
correlations are presented in Table 1, together with the means, 
standard deviations, and intern<J consistency alpha 
reliabilities. Results are entered below the main diagonal for 
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the meles, and above the main diar^onal for fermles, 



Insert Table 1 about here 



Criterion 1 , Convergent validities were all statistically 
significant (£ <.05; mean r==.60) for both nnles and females. 

Criterion 2 > Convergeat validities were consistently higher 
than correlations between different traits assessed by 
different methods (heterotrait-liateromethod trians^les) for both 
males (34 of 36 comparisons) and females (35 of 36 
compar isons) . 

Criterion 3 , Although convergent validities, for the most 
part, were consistently higher than correlations between 
different traits measured by the sane nethod (heterotrait- 
mononethod triangles) for fenales (13 of 18 comparisons), this 
was not so males; only 10 of the IS comparisons met this 
criterion. In particular, the semantic differential and Guttman 
scales exhibited some method bias; whereas tho former was 
narginally stronger for males, the reverse was true for 
females . 

Criterion 4 « Within each sex, tho pattern of correlations 
among the different traits was fairly similar across methods. 
Across gener, however, ASC differed in its relation with l^SC 
and MSC; whereas ASC correlated hip,her with MSC than with ESC 
for males, the latter relation was almost equivalent for 
fenales . 
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Analysis of Variance 

As an alternative to the Campbal 1-Fiske approach to 
analyzing MTMM data, Kavanagh et al. (1971) proposed an 
incomplete 3-way analysis of variance (ANOVA) procedure that 
decomposes the matrix elements into four sources of variance: 
(a) subjects variance, the overall agreement of subjects over 
all traits (i.e.* convergent validity), (b) subjects by traits 
variance, the extent to which subjects are differentiated by 
various trait measures (i.e., discriminant validity related lo 
traits), (c) subjects by methods, the extent to which method 
effects irrjpinge upon the methods used (i.e., discriminant 
validity related to methods), and (d) error variance, the 
subjects by traits by methods interaction. As such, subjects 
are considered aii random, while traits and methods are 
considered as fixed. Furthermore, an index can be computed for 
determining the relative contribution of each variance 
component to the total variation attributable to convergent and 
discriminant validity, method effects, and error; the index can 
be standariZv^d for jjurposes of MTMM comparisons ac^^oss groups. 
The ANOVA results are presented in Table 2 



Insert Table 2 aboi»t here 



In reviewing these findings, wc see an interesting 
difff}rence between adolescent males and females with respect to 
the main effect for subjects, and the subjects hy traits 
interaction, respectively. Although both were modorately strong 
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for bot\i sexes, subjects accounted for slightly mare vririance 
for males (*43) than for females (.37), while the reverse wns 
true for th^ subjects by traits interaction (fenalcs = .45; 
males • .37). These results indicate that while adolescent 
males were better discrimiTi':ited by the trait-method 
combinations employed than were females, discrinination between 
SC facets was slightly higher foi females, than for males. 
Nonetheless, for both sexes, these two components accounted for 
the largest portion of variance within the analyses. 
Interestingly, findings bearing on the subjects by methods 
interaction for both sexes indicated negligible effects due to 
method effects (rales « .04; females .05); these results 
suggest no operc^tion of method bias within any r.^ the three 
methods employed, for either sex. 

Although the above findings suggested some differences 
across gender, direct co'nparison must derive from standardized 
measures since variance components are influenced by within 
gender error. An examination of these standardized indices, 
however, substantiated the findings related to evidence of 
convergent and discriminant validity, albeit a relatively weak 
method effect appeare'* to operate for both sexes. 
Confirmatory Factor Analysis 

For each sex, the 7-factor model was tested for convergent 
and discriminant validity by means of (a) conparisons with 
alternatively specified models, and (b) examination of 
individual parameter estimates. All CFA analyses were condnrted 
usino LIS-^vEL VT (Joresko^ P< Sorbon, 1985). 



13 



Four r:TMM Approaches 

13 

In covariance structure analysis, the extent to which a 
proposed model fits the observed data is based on the 
livelihood ratio test. However, problens related to ^he 

2 

dependency of X on sample size have led to recommendations 
that the assessment of model fit be based on multiple criteria 
that include a measure of practical fit (Rentier S Bonett, 
1980; Joreskog, 19/1; Joreskog Sorbom, 1985; Marsh & Hocevar, 
1985; Widaman, 1935). This was accomplished in the present 
Study by using (a) the X likelihood ratio, (b) tb . x^/ie%rees 
of freedom (df) ratio, (c) the Bentler & Bonett normed index 
(BBI), (d) T-values and modification indices provided by the 
LISREL VI program, and (e) knowledge of substantive and 
theoretical research in this area. 

To establish various validity criteria, the proposad 
7-factor model was tested against a series of more restrictive 
models in which specifi*: parameters were either eliminated or 
constrained to equal zero. Since the difference in X^ (^X^) is 
itself x^^^is^'^i'^^t^d , with degrees of freedom equal to the 
difTerence in degrees of freedom for the two models, the fit 
differential between comparison models can be tested statist- 
ically. A significant ar^^ues for the superior statistical 
fit of the less restrictive model in the sense that it exhibits 
a lower non-centra lit y parameter (see Saris & Stronkhorst, 
1984). However, despite widespread acceptance and practise of 
this approach to model fitting, researchors are urged to 
exercise caution in implyincy certainty where none exists; 
unless one of the models fits perfectly, any interpretation of 
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a non-zero difference remains problematic • (For a more detailed 
discussion of MTMM model conparisons, see Widaman, 19S5, and of 
model application, see Byrie, 1989; Marsh ^ Hocevar, 19S3), Tn 
addition to nodel comparisons at the matrix level, individual 
parameter estimates for trait and method factor loadings, trait 
intercor relations , method intercorrelations, and estimated 
er ror/uniquenessGS were examined with respect to najjnitude and 
statistical significance. 

Examination of matrices . odness-of-f it indices for the 
series of MTMM models tested for evidence of convergent and 
discriminant validity are presented in Table 3, Model 1 is the 
most restrictive model , representing the null hypothesis that 
each observed measure is an independent factor; it serves as 
the null model against which competing models are compared in 
order to determine the BBI. Models 2-4 represent decreasingly 
restrictive T.^dels, such that Model 4 is the least restrictive, 
having both -orrelated traits and correlated methods. Model 4 
is termed the baseline model since it represents hypothesized 
relations among the traits and methods; typically, it 
demonstrates the best fit to the data. 



Insert Table 3 about here 



For both males and feiales, as expected. Mo del 4 
represented the best fit to the data. Due to known problens of 
esti^iation (see Widanan, 1985), additional fittin't of this 
iiy pothesized model was not conducted; as such. Model 4 included 
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traits and methods, c ch corrolntcd only among thernsclves. (Due 
to technical difficulties with the LTSREL program, trait-nethod 
correlations cannot be tested (Vidanan, 1985)). 

As noted earlier, one criterion related to evidence of 
construct validity is that converge.it validities should be 
substantial end statistically si^.nif icant • Thus, if we conpare 
a model in which traits are specified, with one in which they 

2 

are not, a significant difference in X between the two models 
argues for evidence of convergent validity. As such. Model 4, 
in which traits were freely estirated, was compared with Model 
5 in which no trait factors were specified. As shown in Table 
4. the wr^s highly S3r;nificant for both sexes, albeit 

females demonstrated sonewhat stronger evidence of convergent 
validity than males. 



Insert Table 4 about here 



In a similar manner, we can test for evidence of 
discriminant validity. For exnnple, complete discriminant 
validity of tra.lls argues for th(?ir zero int ercorrela t ions . 
Thus, by comparing a model in which traits are allov;ed to 
correlate freely, with one in which trait correlations are 
constrained to 1.00 (i.e., perfectly correlated), we can test 
for discriminant validity. As such. Model 4, in which traits 
were allowed to correlate freely, was compared with Moc-el 6 
which specified perfect correl a t ion.s among traits. The highly 
significant > as shown in Table 2, again indicated strong 

'i6 
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evidence of trait discriminant validity for both sexes. 

Evidence of method effects (i.e., lack of discriminant 
validity among the methods) can be tested by comparing a model 
in which method factors are freely estimated, with one in which 
no method factors are specified; a significant Ax^ argues for 
the lack of discriminant validity and, thus, for evidence of 
method bias. Accordingly, Model 4 was compared with Model 2 in 
which no method factors were specified. Although this 
comparison yielded statistically significant Ax^ ' s for both 
sexes, the differential values were of much lesser magnitude 
than for the traits. Nonetheless, method discri minabi 1 i t y was 
evident for both males and females. 

Finally, to determine the extent to which each measurement 
scale was contributins to method effects. Model 4 was further 
conpared with three additional models, each of which eliminated 
one of the tliree nethods. Interestingly, for both males and 
females, the Likert scales appeared to contribute least to 
method bias, whereas the Guttman scales contributed the most. 
Overall, method effects appeared comparable across gender (c.f. 
Hamilton, 1931) . 

Examination of parameters . A more precise assessment of 
trait- and method-related variance can be ascertained by 
examining the individual para^neter estimates as specified for 
the hypothesized model (Model 4). Of specific interest are the 
magnitude of the factor loadings and the factor correlations. 
These results are presented in Tables 5 and 6 for males and 
females, respectively. The magnitude of the trait loadings are 
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consistent with earlier convergent validity findings determined 
at the matrix level (see Table A); all loadings were statis- 
tically sigtiificant for both sexes. 

Insert Tables 5 and 6 about here 



Method factor loadings appear relatively similar across 
gender, with two minor exceptions. First, method-related 
variance was slightly higher for males than for females, with 
respect to the Lil:ert measure of general SC (.58 vs, .42) 
Second, nethod loadings related to academic SC consistently 
differed across gender; whereas method variance associated with 
the Likert and semantic differential measures were somewhat 
higher for males (.58, ,41, respectively) than for females 
(•42, .31, respectively), the reverse was true for the Guttman 
measure (males .53; females .62). 

Discriminant validity of traits and methods can be 
evidenced by examining the factor correlation natrices. Results 
generally supported earlier findings reported at the matrix 
level (see Table 2). As such, discriminant validity related to 
the traits tended to be slightly stronger for femnles (mean 
r«.41), than for males (mean r=.50). Nonetheless, all values 
were more than three standard errors from 1.00, indicating 
evidence of discriminant validity. Indeed, ^'arsh and Ilo^evar 
(1933) noted that only when factor correlations approach unity 
should researchers be concerned about a lack of discrir:irani- 
validity witli respect to attitudinal constructs such as SC, 

i8 
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VJidaman (1985) has noted that, despite Campbell and Fiske's 
admonition for the use of maximally dissimilar methods where 
multiple measures are enployed, little attention has been paid 
to the discr iminability of method effects. VJith one exception, 
method correlations (see Tables 5 and 6) indicated only modest 
method discriminability for both males and females. The 
correlation of .87 between the Likert and Guttman scales for 
females was less than 1 standard error from 1.00, indicating a 
strong method effect related to the combination of these two 
methods; all other correlations were more than two standard 
errors from 1.00, supporting evidence of discriminant validity. 
Composite Direct Product Analyses 

As noted above, the estimation of trait-method correlations 
for the CPA model is problematic and not recommended (see 
^'idaman, 1985). Indeed, Campbell and O'Connell (1967, 1082) 
have demonstrated that trait and method factors may interact in 
a multiplicative, rather than an alditive fashion such that 
trait correlations become inflated when both share the same 
method. In other words, the stronger the relation between two 
traits, the more their correlation is likely to increase when 
the method is shared; conversely, independent or near- 
independent traits remain so, despite an identical method of 
measurement. Of importance, then, is that method effects tend 
to exaggerate correlations betv;cen traits that are strongly 
intercorrelated . 

In a seminal paper that addressed this issue, Browne (19SA) 
presented a modified CPA approach designed to accommodate the 
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trait-method interaction effect describee! by Campbell and 
O'Connell (1967, 1982). As such, Browne, based on earlier 
research by* Swain (cited in ^^rovne, 1934), proposed the CDP 
model. The major tenet of this approach to MTMM analyses is 
that the true correlation natrix of observed data is the 
product of trait and method factors. Browne arf^ued that method 
effects act to inflate or deflate relations between particular 
traits • 

In order to test these hypotheses, Browne (1984) developed 

MUTMUM, a computer program that assumes an underlying CDP 

structure of observed MTMM data. Results based on the present 

data were analyzed usinf? this program, and are presented in 

Table 7, The component correlation matrices P and P represent 

T M 

correlations among the four traits and 3 methods, respectively. 
Since the elements in these natrices represent nul t i pi ic a t i ve 
corponent correlations corrected for attenuation, they are 
expected to be higher than correlations between observed scores 
as presented in Table 1 (Browne, 19S4); correlations of 1.00, 
notwithstanding, are very disturbing. These values are 
indicative of boundary parameters, which are automatically 
fixed to l.pO by the MUTMUM program (Coffin, personal 
communication, August, 1989). 



Insert Table 7 about here 



Interpretation of results from the CDP model analyses 
focuses on the trait and method effects as reflected in their 
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respective correlation natrices. To assess the55e effects, 
comparison is made with an oxtrr-'io lodc?! in vM-^rh all trnits or 
all methods' are perfectly correlated; the forner argues for no 
trait effects (i.e., no trait variance), while the latter 
argues for no method effects (i.e., 'lo nethod variance). Thus, 
if the mean correlation in is loss than the nean correlation 
in Pjyj the claim can be nade that, as a group, trait factors are 
impacting to a larger degree than are nethod factors on the 
MTMM matrix elements (Goffin, 1987); the reverse, of course, 
substantiates the greater impact of method factors. Since the 
mean trait correlation for both males (mean r = .30) and 
females (mean r = .57) was less than the niean nethod 
correlation (males, nean r = . 36; females, mean r = .31), this 
finding argues for the superiority of trait effects over nethod 
effects. Nonetheless, while nethod effects appear to be fairly 
weak for both sexes, the impact of trait effects seems to be 
much stronger for females. 

Cudeck (1930) has recently shown that results fron CDP 
mode] analyses can be evaluated in a s trai :;ht.f orward fashion in 
terms of the Canpbell-Fiske criteria. (Since these criteria 
have been described earlier in the paper, t^ey arc not repeated 
here). As such, Criterion 1 may be considered satisfied if all 

A 

elements in are large (i.e., convergent validities are larje 
when methods are highly correlated). Vith one exception related 
to females (r = .44), this criterion seens otherv.ise adequately 
met by both sexes. 

A 

Criterion 2 can be evaluated by exnninin^, the matrix. 
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Cudeck (1988) argues that this criterion is satisfied if (a) 
there are no trait correlations >1.00, and (b) the model seems 
appiopriate' to the data. Given the findins of two correlations 
equal to 1.00 for females, and a totally unacceptable fit to 
the data for both males ( (45) = 1319.46) and females ( 
(45) » 1415.50), the present data failed to meet this criterion 
thus providing insufficient evidence of discriminant validity 
related to SC traits. 

The satisfaction of Criterion 3 argues for the discriminant 
validity of method factors, and hence, the unimportance of 
method effects. Cudeck suggests that if the elements in the 

A 

matrix exceed those in the Pp matrix, this criterion can be 
considered met; indeed, it appears to have been satisfied for 
both males and females. 

Finally, claims that Criterion 4 has been satisfied are 
evidenced by a \v'ell-f i tting model which indicates an accurate 
description of the data (Cudec!;, 1938). Since the fit of the 
CDP model to both the male and female data, as noted above, is 
completely unsatisfactory, we therefore conclude that Criterion 
4 was not net. Indeed, Browne (1984) has noted that, give^i the 
more parsimonious parameterization of the CDP model compared 
with the restricted CFA model, the former is more easily 
rejected by ^oodness-of-f i t tests. Nonetheless, in light of the 
extremely poor model fit for both sexes, the author recommends 
that no substantive interpretations be made from the present 
CDP results; they are therefore included hero for purposes of 
Illustration only. 
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Conclusions 

Four approaches to the analysis of JITMM data conprisinp. 
four traits* and three nethods, have been described and 
illustrated using SC data from hi:^,h school adolescent nales and 
females. Given the elaboration of continpencies associated with 
each methodological procedure, it is hoped that researchers 
will benefit substantially from the applications used here. 
Indeed, by demonstrating the ranf,e of information available, 
the four MTMM approaches presented here should aid researchers 
in (a) assessing the strenp,ths and weaknesses associated with 
each procedure, and (b) selecting the methodological approach 
most appropriate to their own research. Although results 
derived from the CDP model were less than optinal and thus 
require further investigation, the illustration of this 
procedure was considered important since the literature is 
presently void of its application to real (versus sinulated) 
da ta « 

Substantively, the paper has demonstrated that adolescent 
males and females can differ with respect to various aspects of 
construct validity related to multidimensional facets of SC. 
Admittedly , certain findinp,s appeared to be ne thod-speci f ic 
(e.g., discriminant validity related to traits and methods); 
further investigation of these results remain the task of 
future research that is currently underway. Nonetheless, one 
consistent gender difference bears inportantly on the structure 
of the construct itself - the relation between nSC and IISC with 
ASC; whereas for females, ESC is more closely related to ASC 
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than is MSG, the reverse is true for nalos. Indeoc!, thesa 
findings have been repeatedly supported by a number of 
independent' Studies (for a review, see ^!ar3h, in press). Such 
gender difference findings should make researchers and 
practitioners more cognizant of potential psychometric 
differences in the measurement of adolescent SC across gender; 
such differences bear importantly both on the selection of 
assessment instruments and on interpretative meanings assigned 
to their rendered scores. 
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Tablt 1 



i^uttftrait-wultititthod Matrix pf Ztro-ordcr Cor relations Among Self-Conceot Measures ^or Mates and Fefrates * 



Liktrt 

(SOQ no 



Semantic Differential 
(API) 



Guttman 
(SES) (SCAS) 



Neaturt 

MSG 



GSC ASC ESC MSC 



GSC ASC ESC MSC 



GSC 



ASC 



ESC 



Likert 
GSC 
ASC 
ESC 
MSC 

Semantic Differential 

GSC 
ASC 
ESC 
MSC 

Guttman 
GSC 
ASC 
ESC 
MSC 




.77 ^^.34 



.30 



I.25N 7S ^^.34 .-^9 



I 

|.10. .41 .51^^^ .H 
^^21^ .SI .13 .83^ 



.^6 V .52 



.21 
.14 



•3V, 



"Ti? 


"2] 


.40 


.411 


N 






.031 


N 









Pn n 


.25 






1 ^ ^ 








1 .54 ^ .64 ^ ^ 


.38 


.45 } 


.66 


1 X 








J N 








1.17 .^3^^^ 




.14 1 


.27 


• --^ 


JOjS^ 


.88'^ 


.^9 




^§2 ^v57 



.22 



.31 I 
I 



I N 

31 ^^.59 ^^.36 .62 

20 .37 .fi x .201 

25 .47 .UN .til 



.S^v 19' 



.17 



.33 .66 .47 .36 



.34 .a 



.04 



;.3i 

1^16 .38 .03 K 



^68 N^14 .14 .08 1 
N V } 

.57 58 ,35 .321 

I 

.21 .«1 .031 
^.18 .«1 .05 N .81 
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Ttblt 1 (cont^d) 

Hultltralt-wjltttthod Matrix of Z«ro-ordcr correla tiont Among Stlf-Conctot >to>turw for ?iiies and Fawles * 



Lfktrt SeiMntIc Dlfftrtntlal Guttman 

no (API) (SES) (SCAS) 



NMSurt 6SC ASC ESC NSC GSC ASC ESC NSC GSC ASC ESC NSC 
Nalaa (D « 412) 

H 76.41 52.9B 55.62 49.22 78.22 71.17 57.92 47.40 32.02 28.36 26.82 26.24 

a 13.86 13.38 10.02 15.98 9.44 10.00 11.09 11.76 4.89 5.94 5.93 7.87 

2 .94 . 89 .78 .92 .86 .85 .88 .95 .85 .88 .90 .94 

F«Mlta {Q * 420) 

H 75.35 57.62 57.93 44.42 75.34 74.15 63.17 43.77 30.73 28.89 28.87 24.24 

a 14.52 11.01 9.47 16.34 8.85 8.65 10.70 11.07 5.02 4.87 5.32 7.09 

2 .94 .88 .79 .93 .86 .83 .88 .94 .88 .85 .89 .94 

' Corrtlationa for mi— art btlow the main diagonal, and for foaalet, abovt the Min diagonal. AU correlationa > .07 are 
aignif icant (p < .05). , 

Note: The underlined valuee are convergent validities. The values in solid triangles are discriiainant validities 
(heterotrait-MnoMthod correlations); those in broken triangles are discrininani: validities (heterotrait-hetero- 
Method correlations). 

a « alpha relfability coefficient; CSC ■ general self -concept (SC); ASC « acadeaiic SC; ESC « Er^lish SC; NSC « 
mathMtics SC; SDQ III ■ Stlf Description Questionnaire III; API ■ Affective Perception Inventory; SCAS « SC of 
AbJlity Scale. 
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Tiblt 2 

AnalytU of Varfanct DaconpoaUion of the Nultltrait-multimethod Matrix for Males and for Femnles 



Source of Variance SS df MS F Variance Standardized 

Conponent Comparison Index 



Males (n = 412) 

Subjects (A) 2234.69 411 5.44 20.92^* .43 .62 
(Conver9ence) 

Ax Traits (B) 1700.74 1233 1.38 5.31 .37 .59 
(ditcrini nation) 

A X Methods (C) 360.91 822 .44 1.69 .04 .13 
(Mthod/halo) 

A X • X C 647.66 2466 .26 .26 



Females (n « 420) 

Sii>jocts (A) 1975.68 419 4.72 16.74 .37 .58 
(convergence) 

Ax Traits (B) 2021.04 1257 1.61 5.96 .45 .63 
(discrinf nation) 

Ax Methods (C) 372.96 838 .45 1.65 .05 .16 
(nethod/halo) 

A X B X C 670.32 2514 .27 .27 
(error) 



Note ; All F- values are significant at p < .001. 
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Males Fenales 
(n = 412) (n = 420) 



Model df BBI df BBI 

1 12 unoorrelatsd factors 3740.55 66 — 3598.79 66 — 
(null model) 

2 4 oarrelated trait factors 474.43 48 .873 430.44 48 .880 
no nethcd f 2K:tors 

3 4 oorxelated trait factors 237.04 36 .937 199.03 37 .945 
3 'Jixxurrelated method factors 

4 4 oorxelated trait factors 182.70 34 .951 126.37 34 .965 
3 oorxelated method factors 

(baseline model) 

5 no trait factors 1867.18 52 . 501 2251.15 52 . 374 
3 correlated nethcxi factors 

6 4 perfectly oorxelated trait 1151.27 40 .692 1338.72 40 .628 
factors, freely oorxelated 

method factors 

7 4 oorxelated trait factors 243.22 40 .935 189.12 40 .947 
2 oorxelated methcxl factors 

(seoantic differential, Guttman) 

8 4 oorxelated trait factors 273.75 40 .927 215.86 40 .940 
2 oorxelated method factors 

(lijoert, GuttsBn) 

9 4 correlated trait factors 362.88 40 .903 309.53 40 .914 
2 oorxelated method factors 



(Likert, semantic differential) 
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Table 4 

Gcx3dness-of-f It Indloes for OcmMrison of Miltit rait-nultiinethod Mcxtels 



Msdes Feaooales 
(n = 412) (n « 420) 



Differenoes in Differenoes in 

HDdel OGn{>ariscn df BHI df BHI 



Tests of Ackled Ocxqpcnents 



Model 1 vs Mcxiel 2 


3266.12 


18 




3168.35 


18 




NDdel 2 vs Model 3 


237.39 


12 


.064 


231.41 


11 


.065 


Model 3 vs Model 4 


54.34 


2 


.014 


72.66 


3 


.020 


Test of Oonvergent Validity 














Model 4 vs Model 5 (traits) 


1684.48 


18 


.450 


2124.78 


18 


.591 


Tests of Discriminant Validity 














Model 4 vs Model 6 (traits) 


968.57 


6 


.191 


1212.35 


6 


.337 


Model 4 vs Model 2 (methods) 


291.73 


14 


.078 


304.07 


14 


.085 


Tests of Method Bias 














Model 4 vs Model 7 (LiJoert) 


60.52 


6 


.016 


62.75 


6 


.018 


Model 4 vs Model 8 


91.05 


6 


.024 


89.49 


6 


.025 


(saiiantic differentied) 














Model 4 vs Model 9 (Guttnan) 


180.18 


6 


.048 


183.16 


6 


.051 



All differencae values are statistically significant (p < .001) . 
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Table 5 



FKtor and Error/Uniajm-t Loadinoa. and Factor Cor relttions for Baseline uodmi umi^ ^ 



Trait 



Method 



Measure 



Likert 

ganeral SC 

acadanic SC .0 

English SC .0 

Mthanatics SC .0 



Saaantic Differential 



81* (.05) .0 



.80* f.04) 

.0 

.0 



.0 
.0 

.80* (.0^) 
.0 



.0 
.0 

.0 

.94* (.04) 



.58* (.12) 

.OS (.OS) 

.13 (.OS) 

.01 (.03) 



11 



.0 
.0 
.0 
.0 



III 



Error/ 
Uniqueness 



.02 (.13) 

.36* (.03) 

.36 (.04) 

.11* (.02) 



ganeral SC 
acadeMic SC 

English SC 
MthSMtics SC 

Guttaan 



.66* 

.0 
.0 

-0 



(.05) 



.78* f 04) 

.0 

.0 



.0 

.89* (.04) 
.0 



.0 
.0 

.94* (.04) 



.0 
.0 
.0 
.0 



.57* (.07) 
.41* (.06) 
.14 (.04) 
.05 (.03) 



.0 
.0 
.0 
.0 



.22* (.07) 
.22* (.05) 
.17* (.03) 
.12 (.02^ 



ganeral SC 
acadaaiic SC 
English SC 
matheaiatics SC 



.81* (.05) 

.0 

.0 

.0 



.0 

.82* (.04) 

.0 

.0 



.0 
.0 

.72* (.04) 
.0 



.0 

.0 
.0 

.86* (.04) 



.33* (.06) 

.53* (.04) 

.32* (.04) 

.19* (.03) 



.27* (.05) 

.00 (.00)^ 

.40* (.03) 

.20* (.02) 



Factor Correlations 



Trait 1 


1.0 












Trait 2 


.72* (.04) 


1.0 










Trait 3 


.29* (.05) 


.58* (.04) 


1.0 








Trait 4 


.42* C.05) 


.74* (.03) 


.24 (.05) 


1.0 






Method I 


•0 


.0 


.0 


.0 


1.0 




Method II 


.0 


.0 


.0 


.0 


-.30* (.12) 


1.0 


Method III 


.0 


.0 


.0 


.0 


.67* (.14) 


-.36*(.09) 1.0 



All values of 1.0 and .0 are fixed values. 



Parameter fixed to .001. 

All parameter eatimates differing significantly from zero are asterisked. Parenthesized values are standard 
errors. 

SC > self-concept. 
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TabU 6 

Ftctor and ErrorAmioiWWM Loadinoa. B rd Factor Correlations for toaeMr»> Mod#l 







Trait 










Method 




Error/ 




















Uniqueness 


Itaasurt 


1 


2 


3 




4 


I 


II 


III 


Lik«rt 




















fltntral SC 


.84* (.04) 


n 
. u 


.0 




.0 


.42* (.09) 


.0 


.0 


.12 (.06) 


•cadMfc SC 


.0 


.73* (.04) 


.0 




.0 


.15 (.06) 


.0 


.0 


.43* (.03) 


Er^Uth SC 


.0 


.0 


.78* 


(-04) 


-0 


.11 (.05) 


.0 


.0 


.39* (.04) 


MthiMtict SC 


.0 


.0 


.0 




-93* (.04) 


.01 (.04) 


.0 


.0 


.14 (.02) 


Smntic Differential 


















general SC 


.67* (.05) 


.0 


.0 




.0 


.0 


.63* (.0^) 


.0 


.15 (.09) 


acadenic SC 


.0 


.77* (.04) 


.0 




.0 


.0 


.31* (.06) 


.0 


.30 (.05) 


English SC 


.0 


.0 


.85* 


(.04) 


.0 


.0 


.06 (.04) 


.0 


.27 (.03) 


nathenatics SC 


.0 


.0 


.0 




.92* (.04) 


.0 


.06 (.03) 


.0 


-15 (.Oa 


Guttman 




















general SC 


.86* (.04) 


.0 


.0 




.0 


.0 


.0 


.31* (.06) 


.18 (.04) 


academic SC 


.0 


.75* (.05) 


.0 




.0 


.0 


.0 


.62* (.05) 


.00 (.00)^ 


Engl ish SC 


.0 


.0 


.77* 


(-04) 


.0 


.0 


.0 


.29* (.04) 


.31* (.03) 


mathematics SC 


.0 


.0 


.0 




.87* (.04) 


.0 


.0 


.19* (.03) 


.18 (.02) 


Factor Correlations 


Trait 1 


1.0 


















ait 2 


.69* (.04) 


1.0 
















Trait 3 


.36* (.05) 


.65* (.04) 


1.0 














Trait 4 


.22* (^.05) 


.52* (.04) 


-.01 


(-05) 


1.0 










Method I 


■0 / 


.0 


.0 




.0 


1.0 








Netnod n 


.0 


.0 


.0 




.0 


.33* (.15) 


1.0 






Method Iir 


.0 


.0 


.0 




.0 


-.87* (.11) 


-.56*(.11) 


1.0 





a 

All values of 1.0 and .0 are fixed values. 



Parameter fixed to .001. 

All parameter estimates differing significantly from zero are asterisked. Parenthesized values are 
standard errors. 
SC ■ self -concept. 
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Ttblt 7 



Cow)Otitt Pirtct ^rodjct Model Coiwponcnt Correlati on Matrices for Hales and Fenales (P j) 



Trafts Neltt Females 

Traits X Traits 

GSC 1.00 1.00 

ASC 1.00 1.00 1.00 1.00 

BSC .75 .98 1.00 .67 .61 1.00 

^ .34 .75 .97 1.00 -.01 .17 1.00 1.00 

Methods X Methods 

UlC 1.00 1.00 

SO .86 1.00 1.00 1.00 

GUTT 1.00 .73 1.00 1.00 .44 1.00 



GSC ■ general self-concept 

ASC ■ acadenic self -concept 

ESC ■ English self-concept 

MSC * Mthcnatics self-concept 

LIK ' Likert scale 

SO > semantic differential scale 

GUTT " CkJttman scale 



